Throughout history, man has always wanted to be somewhere he is not. Even today, people struggle with the problems that can only be solved by xe2x80x9cbeing therexe2x80x9d even when xe2x80x9cbeing therexe2x80x9d involves substantial danger (ex: bomb squads) or great inconvenience (travel). Existing robotics equipment and processes provide means for a user to direct the actions of remote actuators. Salcudean et al U.S. Pat. No. 5,382,885 issued January 1995 discloses a local tool that can be manipulated causing an operatively identical remote tool such as a scalpel to follow suit additionally providing responsive and even leveraged pressure back on the tool the user grasps in his hand. Response feedback is limited to the moment of the tool itself, not the user""s fingers, hand, wrist, etc. Slotine et al U.S. Pat. No. 5,266,875 issued May 1991 provides feedback in a joystick and additionally provides a transmission delay effect softening process the latter of which can be optionally applicable to ancillary features of the current invention. Tsuchibashi U.S. Pat. No. 5,105,367 issued April 1992 directs a remote arm with a joystick and also provides a camera at the remote location providing feedback to a video screen in front of the user. Desai U.S. Pat. No. 5,587,634 issued December 1996 discloses a body actuated control apparatus and system for commercial sewing machines wherein the user pushes a lever with the knee or leg that directs nearby equipment that is visually observable by the user.
The current invention allows a human operator to perform delicate or indelicate tasks with great dexterity remotely with broad multiple and synchronized simultaneous sensory interaction as if the user were actually there. This includes the accurate, full-body interactive, real time perceptions and control of weight, distance, inner ear balance, motion/inertia, speed, pressure, vibration, impact, resistance to action, sound and a true, position responsive visual interface as if his body were actually in the remote location. Unlike the existing screen displays that reflect a camera view of a remote location dislocated from the actual perspective of the user""s body attitude or user action, the visual perspective of the current invention is responsive to the exact pitch, roll and yaw of the user""s head and body as well as to the relative velocity of the full body. Thus the user actually looks down where his hands really are, sees hands and remote objects in them just as if he were there. Further, the user is not holding a local tool that is interfaced to a singular remote tool that emulates the motion of the local tool but, instead, the local user is actually holding empty air despite the fact that he sees and feels the weight, resistance and vibration of the tool in his hand. Guided naturally by visual, sound and other sensory interaction, the user""s hands, arms and other limbs are acting in the remote world using any tool that can be found at the remote site or just using the hands and other body parts alone. Thus the current invention is not limited to special tools, limited work areas, narrow visual perspective or the narrow scope of remote capabilities of existing technologies. Additionally, the current invention provides the user the ability to feel what he is doing and not just resistance pressure but full 360xc3x97360 degree inertial and gravitational feedback. This allows him to apply appropriate 3 dimensionally precise pressure (ex: in surgery where the remote tool is a scalpel), weight, etc. Further, the current invention is scalable allowing the remote robot to be very large or small while still providing realistic and appropriate feedback to the normally sized operating user. The user will have the very natural perception of being whatever size the remote equipment requires. Further, unlike existing robotic control technologies, the user is not only controlling but is uniquely and in every part of his body, himself physically controlled, even forced, to exist within the real conditions and constraints of a distant environment even as he simultaneously controls the elements of that distant environment.
Thus the objects of the current invention include the provision of coordinated user sensory support and enforced fidelity of synchronized positions while directing the efforts of the remote robot as easily as if the user was in the robot""s place. As an example of these objects, the local user sees and feels right in front of him not only the right hand precisely where his right hand actually is but he also sees and feels, for example, a scalpel grasped just tightly enough in it and the other hand squeezing a human heart just enough to distend the surface but not enough to injure a valve.
It is another object of the current invention to provide a remote tool control apparatus and process that is not limited to a singular tool or a controlled work area. It can be used with any tool of any shape or size that the user is capable of directing with any and all parts of his body which allows the user to move around the object working around a full 360 degrees by 360 degrees and can be moved any distance from the original point of contact as required by the task . . . . In one hand the surgeon feels the texture, stiffness, slipperiness, resistance to squeezing, the pulse as the heart pumps and the weight of the heart just as he feels in the other hand the resistance and vibration of the scalpel as it makes an incision and encounters more resistance as it goes deeper. He appropriately responds to a sudden resistance with just enough additional force to overcome the scalpel""s resistance but not enough to puncture below or overextend the cut. If he needs to follow the patient who is rolled to another floor in another building and use different tools at that new location, he simply walks, following the remote patient to the new building, orients himself to accommodate the new patient orientation, location and equipment, and proceeds to work.
It is anther object of the current invention to provide physical support and interaction for the user exactly responsive to actual objects at the remote site. While the above surgery is going on and the user is intensely working with the scalpel in one hand, a living heart in the other hand and both supported by a rolling tray support upon which the surgeon is leaning heavily upon for support and stability, someone standing next to the user would see the user with nothing in his hands and leaning upon nothing. No heart, no scalpel and no support tray. But the user sees and feels an object in his hand, feels the weight and shape of it and can manipulate the phantom object naturally and easily even though the object he sees, feels and changes and the other objects upon which he rests or climbs upon are actually many miles away. Whatever the local user does to that remote phantom item that is held and felt in his remote hand will be done at the remote location. Thus the current invention requires little or no training of the user to perform intricate remote tasks because there are no device control techniques to learn. The user simply perceives that he is there and his normal senses support that illusion.
It is another object of the current invention to provide an apparatus and process for the user to xe2x80x9ctravelxe2x80x9d (direct the remote) over any distance and over any terrain without any need for laboratory conditions or controlled terrain at the remote and without the need for xe2x80x9ctravel distancexe2x80x9d or props of any kind at the user""s location. The user can travel any distance through the remote while locally remaining in a very small space.
It is another object of the current invention to provide an apparatus and process allowing the user to be unencumbered in what he can do and unlimited in what dimensions he can move while directing the remote robotic equipment to perform any action a normal person can do. Thus the user can jump over a log or perform any number of somersaults without worrying about becoming tangled up in equipment.
It is another object of the current invention to provide, without modifications to the local equipment, the ability to direct remote equipment that is proportionately larger for heavy industrial or other applications where power is an issue. Also, using the xe2x80x9cMicromanxe2x80x9d option, the user directs, responds to and is constrained to react naturally in a much smaller environment. Thus the user can stroll into a mouse hole or into delicate equipment to make repairs. Or the user can direct very large robotic responses for heavy industrial work with the natural feeling of being the size of the remote equipment.
It is another object of the current invention to provide the user with further and natural extended sensory perception including body acceleration/inertia, weight of his own limbs in any number of positions, balance (with full inner ear response) and all the natural responses relative to the remote robotic equipment""s pitch, roll and change in yaw.
It is another object of the current invention to provide a means for complete body coordinated responses to remote conditions allowing the user to easily control the myriad of rapidly changing remote environmental conditions with the natural, yet extremely complex to recreate or guide with direction devices, reactions to remote conditions. Impact, for example, of a forward outstretched arm encountering a wall not only extends the user""s fingers backwards, tilts the wrist, bends the arm, etc. but also causes the waist to give and the head to move down and forward providing the user not only the sensory perception of the remote robotic equipment""s condition but the precise operational perspective (dimensionally responsive visual feedback, balance, etc.) to support the quick and natural response remote control maintenance that is lacking in prior art.
It is another object of the current invention to enforce complete body synchronization with the remote equipment in a manner that, even while allowing the user sensitive directive control, allows the user to feel and respond precisely to remote physical conditions, climb a locally non-existent mountain (it""s at the remote site), hang from a locally non-existent tree limb or lie down horizontally on a non-existent bed enjoying the remote bed""s exact body support even though there is nothing below the user.
It is another object of the current invention to provide a simple mechanism with programmatic control that allows the equipment itself and, optionally, the user himself, to appear weightless and thus able, for example, to extend an arm indefinitely without tiring.
It is another object of the current invention to provide a user equipment design that can be automatically and/or programmatically adjustable to the body of different users rather than requiring retooling each time a different person uses the equipment. It is another object of the current invention to provide a design means that allows the remote robotic equipment to adjust manually or programmatically to match the body length and weight distribution of the human user.
It is a further object of the current invention to provide a unique exoskeletal design that provides natural response without the requirement of overly cumbersome devices clamped on the user. It is also an object of this invention to provide an exoskeletal control design that simultaneously measures and enforces extremely complex body attitude combinations with extraordinarily minimal data transmission or processing allowing smooth response cycles.
It is also an object of the current invention to, in addition to communicating to the user physical impact and regional vibration through the responsive and synchronized joint control apparatus, provide the user a sense of touch and vibration allowing the user to better xe2x80x9cfeelxe2x80x9d the remote environment.
It is another object of the current invention to provide a means for allowing the user equipment to be effectively and even automatically adjustable so that it effectively matches the desired alignment, tightness, looseness, etc. of every joint in the robot even as the equipment on both ends tightens or loosens with wear over time.
It is another object of the current invention to provide 3-dimensional, 360 degree by 360 degree relative sound response providing the user with direction and amplitude feedback precisely responsive to the attitude of the remote equipment""s relative 3 dimensional attitude towards the sound sources. In addition to the larger effects, this can include even the refinements of outer ear shape and position as sounds from different directions are focused by the oval sound-shell of the outer ear.
It is another object of the current invention to provide an apparatus and process allowing the user to operate with all the abilities as described above but in a completely virtual environment rather than directing a remote physical set of robotics equipment. This includes the user swinging on virtual tree limbs, shaping and tooling virtual objects for later automated production and testing equipment that has been designed in and exists only in a virtual environment like the one documented in the patent application xe2x80x9cRapid Setup Universal Simulator Predicting Complex Physical Outcomesxe2x80x9d.
It is also an object of the current invention to provide a means to rapidly set up a simulated environment superior to existing simulator sets that require massive custom programming for each simulated environment, extensive physical sets, motorized seats, surround screens, custom designed furniture and furnishings. The simulation environment can be quickly assembled virtually by xe2x80x9cdragging and droppingxe2x80x9d objects from from 3-D CAD drawings or previous simulations as described in the patent xe2x80x9cRapid Setup Universal Simulator Predicting Complex Physical Outcomesxe2x80x9d and the user (and any number of other simultaneous users) can immediately walk, climb, operate and test equipment, etc. in a highly complex simulator that is completely virtual.
Although these abilities have great application to numerous and pressing needs in manufacturing, engineering design and defense, they have previously not been possible without the new elements in the current invention.